Mental Load and Fatigue Assessment Instruments: A Systematic Review

Mental load and fatigue are important causes of performance decreases and accidents in different activities. However, a robust systematic review, detailing the instruments used to quantify them, is currently lacking. The purpose of this study was to summarize and classify by derivations the validated instruments used to quantify mental load and fatigue. The most representative electronic databases in the scope of this review, PubMed, WOS, Scopus, SPORTDiscus, and PsycINFO (until September 2020) were searched for studies that included instruments to analyze mental load and fatigue. The quality of the selected studies was scored using a quality assessment checklist. A total of 40 papers were included. Most of the papers used subjective scales (75%) to quantify mental load and fatigue, with a small presence of behavioral (n = 5) and objective techniques (n = 5). Less is known about the analysis of mental load and fatigue using a combination of derivations. Despite the high cost and complexity of objective techniques, research that applies these measures is important for further analysis of brain processes in mental load and fatigue. The design of a battery of tests that include the three types of derivations also seems necessary.


Introduction
Fatigue can be caused by excessive mental and/or physical demands, but the analysis of fatigue has focused on physical aspects [1]. Physical fatigue causes impairments in the traditional physiological variables (i.e., heartrate, blood lactate, or oxygen uptake). Contrary to physical fatigue, mental fatigue is not associated with these impairments, although the specific role of the brain has been demonstrated in mental fatigue [2]. Mental fatigue is apparently caused by excessive mental demands [3], and some authors have used the term "mental load" to refer to it [4]. Mental load and fatigue have been widely associated with specific performance decreases or an increase in the risk of accidents [1,5]. Although the impairments caused by mental aspects reveal the importance of quantifying these variables, a robust systematic review of the available instruments used to quantify mental load and fatigue is currently lacking. This information may enhance the importance of controlling these variables and facilitate experts' choices of the most adequate instrument according to their needs.
Mental fatigue represents a psychobiological state with subjective (e.g., an increase in feelings of tiredness), behavioral (e.g., motivation decrease or reaction time increase), and physiological (e.g., alterations in the electroencephalogram signal) derivations in humans [6]. This psychobiological state is caused by brain-demanding tasks (i.e., mental load), with a relevant role of emotional (e.g., anxiety or stress) and cognitive (e.g., working memory or cognitive flexibility) aspects [6]. This should be considered in the analysis of mental load and fatigue, although most of the studies performed have used the cognitive aspects of mental fatigue [6]. For example, the case of Simon Biles or the different episodes observed during the COVID- 19 are examples of how emotional aspects may impair health and performance.
Specifically, it has been observed that mental load and fatigue may impair human physical performance [6]. Some authors have stated that this phenomenon occurs through the increase in the subjective Ratio of Perceived Exertion, whereas other performance indicators, such as accuracy, tactical decisions, or reaction time, may be impaired by excessive accumulation of mental fatigue [1]. These impairments have been observed in different contexts such as medical surgery, construction work, or athletic settings [1]. Although the accumulation of extracellular adenosine or impairments in cognitive functions are possible explanations of this phenomenon, more studies are necessary to clarify the causes underlying these impairments.
However, it is difficult to analyze the causes and consequences of mental fatigue. Several covariables influence the mental fatigue induced by task performance, such as task difficulty, engagement, duration, or enjoyment/aversion [7]. In addition, a large number of individual differences could explain why the same task does not induce the same level of mental fatigue in different subjects or why mental fatigue manifests with different derivations (i.e., subjective, behavioral, or physiological) among participants [8]. Van Cutsem and Marcora [7] strongly recommend the use of a combination of several derivations (i.e., subjective, behavioral, and physiological) of mental fatigue as the best approach to identify its presence. Changes in all three areas do not necessarily appear in mentally fatiguing conditions, and they could depend on the subjects' individual characteristics. For example, cognitive performance does not necessarily decline in presence of mental fatigue due to the effect of the compensatory effort system [6][7][8][9]. Therefore, the use of different measures of mental fatigue may identify the causes of mental fatigue or explain why mental fatigue impairs performance.
Despite these recommendations, few existing procedures allow experts to assess mental load and fatigue, making them difficult to control [3]. On the one hand, different instruments have been used for this purpose indirectly, subjectively, and behaviorally. For mental load, we find the (i) NASA Task Load Index [10], (ii) the Subjective Mental Workload Scale (SCAM) developed by Ceballos-Vásquez et al. [11], or (iii) the "StuMMBE-Q" [12], among others. For mental fatigue, the subjectively reported Visual Analogue Scale (VAS) has been the most used instrument. Despite the high reliability and validity of these instruments, information about brain processes is lacking. On the other hand, objective instruments have also been used for this purpose. Pupil dilation [13], eye tracking [10], and different electrophysiological indicators such as electroencephalography (EEG; [14]) or brain functional connectivity patterns [15] have been recommended by authors to quantify mental load and fatigue. Although these measures are necessary to increase the quality of the data of mental fatigue, the high cost and the low ecological validity of the data extracted (e.g., EEG requires a sedentary activity to perform the measures; therefore, in a sport-specific context, experts cannot quantify the mental activity) of many of these instruments makes their use difficult.

The Present Study
Consequently, although the importance of mental variables in daily activities, work, or sports has increased because of the negative consequences of mental load and fatigue, it is difficult to choose a valid instrument to assess mental load and fatigue considering the different derivations caused by mental fatigue. Therefore, the first research question of this study is: What instruments exist to quantify mental load and mental fatigue? Consequently, the main purpose of this study was to summarize the instruments used to quantify mental load and fatigue. The second research question is: What instruments are more adequate to quantify each specific derivation of mental fatigue? Hence, we have also classified them by the type of derivation quantified to allow experts to choose the most adequate single instrument or battery of instruments, following the recommendations of using a combination of instruments for different derivations.

Materials and Methods
With this systematic review, we methodologically and comprehensively searched, appraised, and synthesized research evidence [16] for studies, aiming to identify the instruments used to quantify mental load and fatigue. This research was developed following the Preferred Report Elements for systematic reviews and meta-analyses (PRISMA) recommendations for systematic reviews [17] and the elements chosen for review [18]. Furthermore, this review was preregistered using the international prospective register of Systematic Reviews and Meta-analysis (PROSPERO [19,20]; registration: CRD42020167775).

Eligibility Criteria and Search Strategy
We followed the systematic review procedure suggested by Grant and Booth [16]. We included original empirical research papers published each year until September 2020 (i.e., we did not specify the start of the year, including all articles published until September 2020). Papers selected for analysis were found through searches of the most representative electronic databases for the scope of this review: PsycINFO, PubMed, Scopus, SPORTDiscus, and Web of Science. To identify the studies that used instruments to analyze mental load and mental fatigue, the authors used broad inclusion criteria, and all relevant research was included in the present study [21]. The following search terms were used to explore electronic journals: (i) mental load or mental fatigue; (ii) assessment OR measurement OR instrument; and (iii) validation. In each of the databases, the advanced search option was used to obtain the best combination and to access all possible research within our study framework. For example, in the Web of Science database, the following search was performed: TS = ("mental load" OR "mental fatigue") AND TS = ("assessment" OR "measurement" OR "instrument") AND TS = ("validation"). In addition to the search carried out in the databases, we performed a manual search to identify additional works to include in the study.
Before beginning the investigation, the inclusion and exclusion criteria were established to correctly define the objectives of this systematic review. Considering the search terms, we decided to include all the available works in each database, including all the languages present in each investigation. In addition, all articles published at any time before September 2020 were included. Another inclusion criterion was that all documents were original, with the full text available for analysis [22]. Articles with some measure or validity instrument on mental load or mental fatigue were also included.

Study Selection and Data Collection Process
Within the screening system, after reading the title and summary, the full text of the articles that were considered suitable for the review was selected to be evaluated and introduced into the study. The PRISMA flowchart ( Figure 1) represents the filtering system for the final collection of the selected sample to complete the preparation of the current review. An expert meeting was held at each of the filters to determine the inclusion or exclusion of the different works previously analyzed. Figure 1 also shows the number of documents included and excluded in each of the phases of the screening process.
At the end of the search in the different databases, a total of 327 potential studies was obtained, of which 94 were eliminated for duplication and 158 due to the topic. Subsequently, 21 studies were eliminated for lack of the full text, and 14 because they did not meet the established quality criteria. After this selection process, a total of 40 articles was obtained.
All these steps were performed independently by two researchers following the same criteria. Kappa statistic (k) was employed to test the percentage of interrater agreement, indicating strong agreement between the two raters (k = 0.85, [23,24]). Discrepancies were discussed with a third reviewer until 100% consensus was reached. At the end of the search in the different databases, a total of 327 potential studies was obtained, of which 94 were eliminated for duplication and 158 due to the topic. Subsequently, 21 studies were eliminated for lack of the full text, and 14 because they did not meet the established quality criteria. After this selection process, a total of 40 articles was obtained.
All these steps were performed independently by two researchers following the same criteria. Kappa statistic (k) was employed to test the percentage of interrater agreement, indicating strong agreement between the two raters (k = 0.85, [23,24]). Discrepancies were discussed with a third reviewer until 100% consensus was reached.

Data Synthesis
Once the definitive studies were selected, a synthesis of the information and the most important characteristics of each article were extracted. Data relating to the instrument used, authors and year of publication, sample characteristics (i.e., number of volunteers, sex, age), and other instruments used for comparison were extracted. The results and conclusions to analyze the validity and reliability were collected. Thus, the studies were overviewed and compared, allowing us to evaluate the current state of research on mental load and mental fatigue assessment, which was divided into different sections. Due to the diversity of derivations through which mental load or mental fatigue was assessed, each document was assigned to one of the three subsequently established categories: (i) mental load and fatigue assessment instruments for subjective derivations; (ii) mental load and fatigue assessment instruments for behavioral derivations; (iii) mental load and fatigue assessment instruments for physiological derivations.

Quality Assessment
The quality of all studies was evaluated using the quantitative assessment tool 'QualSyst' [25]. This validated checklist consists of 14 sections, each assessing a different measurement property (see Table 1). Each item within a section is scored on a three-point scale depending on the degree to which the specific criteria were met (yes = 2, partial = 1,

Data Synthesis
Once the definitive studies were selected, a synthesis of the information and the most important characteristics of each article were extracted. Data relating to the instrument used, authors and year of publication, sample characteristics (i.e., number of volunteers, sex, age), and other instruments used for comparison were extracted. The results and conclusions to analyze the validity and reliability were collected. Thus, the studies were overviewed and compared, allowing us to evaluate the current state of research on mental load and mental fatigue assessment, which was divided into different sections. Due to the diversity of derivations through which mental load or mental fatigue was assessed, each document was assigned to one of the three subsequently established categories: (i) mental load and fatigue assessment instruments for subjective derivations; (ii) mental load and fatigue assessment instruments for behavioral derivations; (iii) mental load and fatigue assessment instruments for physiological derivations.

Quality Assessment
The quality of all studies was evaluated using the quantitative assessment tool 'Qual-Syst' [25]. This validated checklist consists of 14 sections, each assessing a different measurement property (see Table 1). Each item within a section is scored on a three-point scale depending on the degree to which the specific criteria were met (yes = 2, partial = 1, no = 0). A score of >0.75 indicated strong quality, a score between 0.75 and 0.50 indicated moderate quality, and a score <0.50 indicated weak quality. The term "NA" was used for those items without a particular study design, which were excluded from the calculation of the summary score. This process was carried out by two reviewers (M.A.L.G. and J.C.P.B.), and discrepancies were discussed with a third reviewer (J.D.G.) until 100% consensus was reached. Likewise, the Kappa statistic (k) was employed to test the percentage of interrater reliability [26]. These steps were performed by two reviewers. The agreement between researchers reflected in the kappa coefficient (κ = 0.84, k = 0.85) indicated a strong initial agreement between the two raters [23,24]. Regarding quality assessments within individual studies, the kappa coefficient (κ = 0.91) indicated a strong initial agreement between the two raters [23,24]. Quality assess-ment of these 40 selected articles showed that 30 articles were of strong quality, 8 articles were of moderate quality, and 2 articles were of weak quality (see Table 1).  Table 2 shows the studies (n = 31) that have used and tested instruments for the subjective derivation of mental load and fatigue. Of these studies, 75% focused on subjective derivation. However, we observed that most of these instruments focused on terms related to mental load and fatigue but not on these specific terms. These related terms are, for example, job-related stress (e.g., Mental Workload Instrument or Fatigue Assessment Scale for Construction Workers), or chronic fatigue syndrome. Concerning the variables and instruments used to validate these instruments, most of these studies used other related scales such as the Ratio of Perceived Exertion to compare the results obtained. Some authors used the behavioral consequences of mental fatigue, such as sleep (i.e., PSQI), in comparison with the CFS and biological parameters (i.e., ECG). The population used to validate these instruments ranged from school and university students to workers and clinical patients.   Correlation coefficient between RSME and ECG is 0.85. ECG and CBC-MWL measurement show a high correlation with the RSME score.

Mental Load and Mental Fatigue Assessment Instruments for Subjective Derivations
Proposed method is consistent with the RSME method but RSME cannot be completed in real time. Work Ability Index 7.
Utrecht Work Engagement Scale 8.
Work Involvement Scale.
Cronbach's alpha coefficients were calculated for each WRFQ subscale and the total score (ideal, between 0.70 and 0.95). WRFQ 2.0 is a reliable and valid instrument to measure the health-related work functioning in the working population in general. Table 3 presents five studies that used instruments to analyze the behavioral derivations of mental load and fatigue. Of these works, 12.5% focused on these derivations. Cognitive functioning, using attention, eye movement, accuracy, performance drive, or reaction time, was analyzed in this type of derivation of mental fatigue. To validate these instruments for the analysis of the behavioral derivations of mental fatigue, they were compared both with scales and questionnaires (e.g., MFS or CFS), other behavioral consequences (i.e., sleep), and physiological derivations (i.e., EEG). Most of these studies were performed with healthy participants, university students, or clinical patients, showing a higher variety of population than in the validation of the previously analyzed subjective scales.

Mental Load and Fatigue Assessment Instruments for Physiological Derivations
Finally, Table 4 presents a total of five studies that designed a battery of tests or instruments to analyze the physiological derivations of mental fatigue. Of these studies, 12.5% focused on such derivations. The main instrument used for these derivations was the EEG. Concerning the comparison of instruments, some of these investigations used behavioral responses to compare the instruments, for example, a comparison of VAS and EEG. Samples of workers and healthy patients were used in these studies.

Discussion
The aim of the present study was to summarize the different mental load and fatigue assessment instruments used, as well as to show their accuracy, reliability, and validity according to the derivation of mental load or fatigue analyzed by these instruments. The main results showed that there is a prevalence of subjective scales to measure mental load and fatigue. However, the use of electroencephalograms appears as an emergent form to understand the biological mechanisms of mental load and fatigue.

Mental Load and Mental Fatigue Assessment Instruments for Subjective Derivations
Our results showed that 75% of the instruments included in the present study focused on the subjective derivations of mental load and fatigue. These results indicated a tendency to use self-reported questionnaires or scales in the analysis of mental load and fatigue. The extended use of these types of instruments may be explained by the high validity and usefulness of their measurements [63]. However, experts should take into account the context involved to choose the most valid instrument, according to the data to be extracted. Previously, Russell et al. [8] defined the complex nature of human factors, which could explain why, when analyzing mental fatigue, experts also analyzed other psychological factors. Indeed, work settings and hospitals were the main contexts where these instruments have been used, whereas in other contexts, such as schools or sports, where mental fatigue is present [64,65], few papers have analyzed the validity and reliability of these instruments [7]. As mentioned, these types of instruments are useful in the research of students and athletes because these populations usually have little time to answer our research questions [1]. The main interest of these instruments is the individualization of the feelings of mental fatigue [8]. Such individualization of the context is important from a clinical and practical viewpoint. For example, in a sports context, one task may significantly increase the mental fatigue of a certain athlete, while this same task will not change the mental fatigue of another athlete. This may be extended to hospital patients, students, or workers because mental fatigue has a subjective derivation, among others. Indeed, this situation justifies the use of these scales. However, although these types of instruments have highlighted the role of mental fatigue and promoted the study of this variable, a great number of experts have declared that further analysis of the physiological mechanisms is needed to explain mental load and fatigue [1].

Mental Load and Fatigue Assessment Instruments for Behavioral Derivations
Our results show that 12.5% of the instruments included in the present study focused on the behavioral derivations of mental load and fatigue. These variables allow experts to determine how mental fatigue may influence performance indicators in each context. Russell et al. [66] asked an athletic population about their symptoms in the presence of mental fatigue. These athletes felt slower, with poor reaction times and decreased accuracy. Moreover, a great number of papers have demonstrated the relationship between an increase in the feelings of mental fatigue and a decrease in the specific behavioral performance in different areas [34,52,67]. On the contrary, the results of the present study suggest that few instruments have been validated for this purpose from a behavioral perspective. From a clinical and practical point of view, this implies a limitation in the analysis of the negative effects of mental fatigue. Mental fatigue is important because of its negative consequences in surgeons, athletes, or performance and health drivers. More studies designing instruments for behavioral derivations or examining the effects of mental fatigue in human behavior are necessary to further analyze the importance of mental fatigue.

Mental Load and Fatigue Assessment Instruments for Physiological Derivations
Finally, our results show that 12.5% of the instruments included in the present study focused on the physiological derivations of mental load and fatigue. The influence of the brain in mental fatigue has been demonstrated; indeed, this influence has allowed researchers to differentiate the mental and physical nature of fatigue [2]. Whereas physical fatigue is normally caused by an impairment in the traditional physiological systems, such as heartrate or blood lactate, impairments in these systems have not been observed in the performance-related decreases in mental fatigue [2]. This shows that less is known about the psychobiological processes involved in mental fatigue. Although the complexity of these instruments (price, complexity, time . . . ) could explain the few papers published about these instruments, this information would allow researchers to understand the mechanisms that underly the presence of mental fatigue and its consequences [7]. This information is interesting from a clinical and practical viewpoint. For example, it would be useful to know how mental fatigue can be manipulated, how recovery strategies can be used, or how to maintain performance in presence of mental fatigue. Indeed, as mentioned, a large number of experts support the importance of further analysis of this derivation to advance in this topic.

Strengths and Limitations
This investigation presents a series of noteworthy strengths. Firstly, to our knowledge, no previous studies have studied the instrument used to analyze mental load and fatigue. Indeed, no previous studies have classified the instruments used to analyze mental fatigue according to the type of derivation.
The present research also presents some limitations that should be mentioned. The main limitation of this systematic review is the difficulty to obtain definitive conclusions, based on the heterogeneity of the type of instruments (e.g., scales, EEG, questionnaires).

Practical Applications and Future Research
The main practical application of this investigation is that these data could allow experts to choose an adequate instrument to analyze mental fatigue according to their needs. Experts could even design a battery of instruments to analyze mental fatigue from a global perspective.
For future research, we highlight the need to design specific instruments to quantify mental load and fatigue in sports or education. In addition, the use of behavioral and objective measures (e.g., blood sample or EEG) would allow further analysis of the causes and consequences of mental load and fatigue.

Conclusions
Of the studies included, 76% focused on the subjective derivation of mental fatigue. Therefore, we can conclude that most of the existing instruments to analyze mental load and fatigue are subjective questionnaires and scales. Furthermore, 12.5% analyzed the behavioral derivation, and 12.5% analyzed the physiological derivation of mental load and fatigue. Thus, few studies have designed instruments to quantify these variables from behavioral and physiological derivations. The scales have allowed experts to highlight the role of mental fatigue, which is important to assess the individual effect of mental load and fatigue in each subject. However, experts also stress the need to study the mechanisms involved in mental load and fatigue, analyzing the physiological mechanisms. More information is also necessary for sports and schools to analyze mental fatigue because most of these works were carried out with hospital patients and workers.